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Abstract 

Background: Bearing in mind the increasing health expenses and their weight in the Portuguese gross domestic 
product, it is of the utmost importance to evaluate the performance of Primary Health Care providers taking into 
account both efficiency, quality and equity. This paper aims to contribute to a better understanding of the 
performance of Primary Health Care by measuring it in a Portuguese region (Lisbon and Tagus Valley) and identifying 
best practices. It also intends to evaluate the quality and equity provided. 

Methods: For the purpose of measuring the efficiency of the health care centers (ACES) the non-parametric full 
frontier technique of data envelopment analysis (DEA) was adopted. The recent partial frontier method of order-m 
was also used to estimate the influence of exogenous variables on the efficiency of the ACES. The horizontal equity 
was investigated by applying the non-parametric Kruskal-Wallis test with multiple comparisons. Moreover, the quality 
of service was analyzed by using the ratio between the complaints and the total activity of the ACES. 

Results: On the whole, a significant level of inefficiency was observed, although there was a general improvement in 
efficiency between 2009 and 201 0. It was found that nursing was the service with the lowest scores. Concerning the 
horizontal equity, the analysis showed that there is no evidence of relevant disparities between the different 
subregions(NUTS III). Concerning the exogenous variables, the purchasing power, the percentage of patients aged 65 
years old or older and the population size affect the efficiency negatively. 

Conclusions: This research shows that better usage of the available resources and the creation of a learning network 
and dissemination of best practices will contribute to improvements in the efficiency of the ACES while maintaining 
or even improving quality and equity. It was also proved that the market structure does matter when efficiency 
measurement is addressed. 
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Background 

Introduction 

Healthcare is one of the most important areas for citizens. 
It is also an area where countries spend a significant part 
of their resources. For this reason, measuring efficiency 
becomes relevant. In fact, in 2009 it was estimated that the 
waste of financial resources with the Portuguese Health 
System was about 25% of the amount allocated to health 
[1]. Furthermore, in line with other developed countries, 
an increase on health expenses, both public and private, 
has been observed. The growth rate of these expenses 
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even exceeded the growth rate of the GDP and between 
2000 and 2008 it rose almost 50%. This growth largely 
rests on the changes of the Portuguese demographic struc- 
ture [2]. This situation has created a constant concern 
about the sustainability of the Portuguese health system 
[2] so that it becomes necessary to promote efficiency and 
innovation by adapting management practices and using 
financial resources in an optimized way. 

Despite the fact that the Portuguese health care model 
is still very dependent on the secondary and differentiated 
care, the idea that health care systems based on a solid 
structure of Primary Health Care (PHC) are more cost- 
effective is, nowadays, fairly pacific [3]. Due to this fact, 
the Portuguese Primary Health Care has suffered many 
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developments. Additionally, given the current economic 
situation of Portugal, there is an extra need to evaluate 
the performance of the public sector and provide value 
for public money. Health Centers Groups (ACES) pro- 
ductive behavior, and particularly, their efficiency might 
contribute to this purpose. As stated by Jacobs et al. [4], 
the efficiency study of health institutions must be a cen- 
tral objective to reduce the public expenditure. According 
to these authors, the non-parametric frontier technique of 
Data Envelopment Analysis (DEA) might be an extremely 
useful tool for estimating the PHC efficiency. 

Since the mid-1980s, efficiency in health care has been 
studied by many authors, both at national and interna- 
tional levels. The first application of DEA in the health 
sector dates back to 1983, in which Nunamaker and Lewin 
[5] measured the efficiency of the routine nursing service. 
Since then, DEA has been widely used in the assessment 
of health care services efficiency. However, the efficiency 
of PHC has not been a significant priority. Actually, in the 
literature review performed in this research, only 9 studies 
were found until 2002 regarding PHC efficiency [6]. 

The first study on the efficiency of PHC was published 
in 1989, when Sexton et al. [7] evaluated the manage- 
rial efficiency of veterans administration medical centers. 
The research concluded that around one third of the 
total number of health centers were inefficient and that 
the elimination of the referred inefficiency could reduce 
annual costs by 300 millions of dollars. 

After that, some studies were performed, including: 
Huang and McLaughlin [8], Pinillos and Antonanzas [9], 
Linna et al. [10], Akazili et al. [11], Kirigia et al. [12], 
Amado and Santos [13], Sebastian and Lemma [14], 
Halsteinli et al. [15] and Nuti et al. [16]. 

A common feature of most of these studies, given 
the level of health care studied, is the variables chosen. 
The most frequent inputs are the staff and the expen- 
diture, while the most adopted outputs are the different 
kinds of consultations related to PHC. Nevertheless, while 
the oldest studies only focused on efficiency, the most 
recent ones also take into account the factors that affect 
efficiency. 

Concerning quality, the relationship between efficiency 
and quality of care has had mixed results in prior studies. 
For example, Helling et al. [17] discovered that increas- 
ing efficiency would result in higher quality. On the other 
hand, Singaroyan et al. [18] found that improving quality 
of health care may not always lead to efficient operations. 

Regarding horizontal equity, it is concerned with fair- 
ness, which means equal treatment of patients [19]. In the 
healthcare field, it measures whether patients from differ- 
ent groups have similar access to the services they equally 
need. 

In Portugal, there are few studies about the performance 
of PHC, either related to efficiency, quality or equity. 



Most of the time, evaluation in Portuguese public ser- 
vices involves rankings, classifications and targets. In fact, 
Amado and Santos' study [13] is one of the very few that 
uses DEA in this scope. So, this study contributes to the 
literature on several grounds. First, it provides an analy- 
sis of efficiency, equity and quality of the PHC in Portugal, 
which is still an area where big improvements must be 
made. Second, because at the date of the study of Amado 
and Santos [13], the PHC in Portugal was not organized 
as it is now, thus the study of efficiency, equity and qual- 
ity of the ACES of Lisbon and Tagus Valley (LVT) is of the 
utmost importance when taking into account the current 
market structure. 

Primary health care in Portugal 
Overview 

Currently, the Portuguese healthcare system is charac- 
terized by three coexisting, overlapping systems: the 
National Health Service (NHS); public and private insur- 
ance schemes for certain professions and private volun- 
tary health insurance. The majority of the population 
receives health care from the NHS, which was founded 
in 1979 and aims to provide health care, almost free 
at the point of delivery, with universal coverage being 
funded mainly by general taxation. Although centrally 
financed by the Ministry of Health, the NHS com- 
prises five health administrations: North, Centre, Lis- 
bon and Tagus Valley, Alentejo and Algarve. Their main 
responsibilities are the development of strategic guide- 
lines; coordination of all aspects of health care provi- 
sion; supervision of management of hospitals and PHC; 
establishment of agreements and protocols with private 
bodies and development of a long-term care network 
[20]. 

Now, it is important to distinguish between Primary 
Care and PHC. Primary Care provides entry into the 
system for all new needs and problems along with people- 
focused (not disease-oriented) care over time and care 
for all but with very uncommon or unusual conditions. It 
also co-ordinates or integrates care provided elsewhere by 
others [21]. PHC is a conceptual model which refers to 
disease prevention, health promotion, population health, 
and community development within a holistic framework, 
with the aim of providing essential community-focused 
health care [22] . This is the level of Health Care we are 
studying in this article. 

In Portugal, PHC has a history of about 40 years. Since 
there has been a constant need of implement reforms, 
it becomes necessary to understand how PHC evolved 
during the last 40 years. Therefore, it is important to dis- 
tinguish two different periods: the first, between 1971 and 
2004, when the first, second and third generation of health 
centers were created and the second one, from to 2005 up 
to now. 



Ferreira etal. BMC Health Services Research 2013, 13:529 
http://www.biomedcentral.eom/1 472-6963/1 3/529 



Page 3 of 1 7 



From 1971 to 2004 

The first legislation concerning health centers is from 
1971, seven years before the Alma-Alta declaration. The 
"First Generation Health Centers" were created under the 
terms of the Decree-law 412/71. These Health Centers 
were responsible for ensuring public health: vaccination, 
women and childrends health, pregnancy care and health 
authority. At this time, the treatment of acute disease was 
not part of PHC activities. Such health care services were 
provided by the sickness insurance institutions [23] . 

Although seemingly paradoxical, regarding the health 
needs and patients' expectations these two styles were 
complementary. That is the reason why a new legislation 
had been approved in 1983 (Decree-Law 97/83) which 
amended the preceding decree in order to integrate the 
health centers and the sickness insurance institutions, 
thus creating "Second Generation Health Centers". How- 
ever, this process was not totally peaceful, mainly due 
to the significant differences in the physical and human 
heritage. Because of that, consultations, home visits and 
health surveillance did not show the expected improve- 
ment. Additionally, these structures had very limited 
management autonomy [24] . 

To counteract this situation, a structural reorganiza- 
tion of PHC was provided for in 1999 (Decree-Law 
157/99), which established an ideal model of a health 
center, the "Third Generation Health Centers". How- 
ever, in practice the situation of dependence was basi- 
cally the same. This Decree-Law was repealed by the 
Decree-Law 60/2003, which aimed to improve the pri- 
mary health care system by creating a network of health- 
care provision. This network was supposed to improve 
the citizens' evaluation capability. It was also an objec- 
tive of the network to contribute to the reversal of 
the conservative policies and aversion to change appar- 



ently responsible for the inefficiency of the traditional 
health system. Nevertheless, this decree has never been 
implemented. 

Since 2005 

A new phase of the Portuguese health sector started with 
the enactment of the Decree-Law 88/2005 which tem- 
porarily reinstated the one from 1999 and referred to a 
technical group specifically created to prepare the new 
health care reform. Since then the Government has begun 
studying ways of improving efficiency, equity and qual- 
ity of PHC. During this period, some innovative ideas 
have arisen, the creation of ACES is an example of great 
importance. 

By definition, the ACES are amalgamations of resources 
and management structures and are composed of dif- 
ferent functional units. Their mission is to ensure the 
provision of PHC in a particular geographic area, enhanc- 
ing health gains accomplished by Family Health Units 
(USFs) and providing better management structures. Also, 
the development of epidemiological surveillance activi- 
ties, the research on health and the control and evaluation 
of results are inherent to the ACESs mission. According 
to "The Mission for Primary Health Care", many fac- 
tors are taken into account when defining the geographic 
area of influence of the ACES: the number of residents 
that should be between 50,000 and 200,000, the popula- 
tion structure, the aging index and the accessibility to the 
reference hospital. 

Each ACES is led by an Executive Director (DE) and 
consists of a Clinical Council (CC), a Community Council, 
a Management Support Unit (UAG) and five functional 
units, as described in Table 1. Other functional units 
might be considered, if the Health Regional Administra- 
tion (ARS) decides so. 



Table 1 The 5 types of functional units that compose the ACES 


Functional unit 


Description 


USF 


Individual and Family Care Unit. USFs promote the training of multidisciplinary 




teams, comprised by doctors, nurses and administrative staff. USFs allow a closer 




relationship with users through constant and personalized contact. In addition, 




in USFs all the patients have assigned doctors. 


Personalized Healthcare Unit 


Individual and Family Care Unit. In terms of dimension, it is similar to an USF. 


(UCSP) 


However, USFs are regulated by specific legislation, whereas UCSP are bounded 




to rules approved by the Clinic Council. 


Community Care Unit (UCC) 


UCCs provide care to groups with special needs and community interventions. 




UCCs operate in the community and are able of mobilizing skills inherent to other 




functional units, to provide health care through specific interventions. 


Public Health Unit (USP) 


USPs are related to population and environmental and public health. USPs are 




responsible for the planning and divulgation in public health. They are also in 




charge of epidemiological surveillance and manages population-wide programs 




in the domains of prevention, health promotion and protection. 


Shared Assistential Resources Unit 


URAPs provide and enhance specific support and advice to the functional 


(URAP) 


units and health projects of each ACES. Their mission is to support the former 




functional units. 
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Regarding the region of LVT (Figure 1), which corre- 
sponds to 13% of the Portuguese territory including the 
capital Lisbon, it has a population of 3,7 million (34% 
of the total population) and, according to the Centre 
for Regional Dynamics Observation (2009), it represents 
44% of the national GDR In this region, there are 22 
ACES, organized according to the five existing NUTS III 
(subregions): 

• Grande Lisboa: Lisboa Norte (1), Lisboa Oriental (2), 
Lisboa Central (3), Oeiras (4), Odivelas (5), Loures 
(6), Amadora (7), Sintra - Mafra (8), Algueirao - Rio 



de Mouro (9), Cacem - Queluz (10), Cascais (11) and 
Vila Franca de Xira (12); 

• Peninsula de Setubal: Almada (13), Seixal - Sesimbra 
(14), Arco Ribeirinho (15) and Setubal - Palmela (16); 

• Oeste: Oeste Norte (17) and Oeste Sul (18); 

• Medio Tejo: Serra dAire (19) and Zezere (20); 

• Leziria do Tejo: Ribatejo (21) and Leziria II (22). 

In the next chapter, the adopted methodology is pre- 
sented: the non-parametric full frontier technique of DE A 
to study the efficiency of the ACES and the partial frontier 
technique of order-m so as to adjust the results obtained 
to the exogenous environment of each ACES. 





Figure 1 Location of Lisbon and Tagus Valley in the Portuguese 
territory. 



Methods 

Data collection 

In this research, we analyzed the activity of the 22 ACES 
belonging to the LVT, mentioned at the end of Section 
'Background', during the years 2009 and 2010. Most of the 
data used in this study was obtained from the ARS of LVT 
activity reports and the Project SimCidadao report. Both 
of the reports are freely available on-line. The data regard- 
ing staff was obtained from many contacts with the ARS 
of LVT. The use of this data required a previous authoriza- 
tion given by this organization. It is important to highlight 
that some inputs concerning costs were not available and 
that is the reason why most of the inputs are related to the 
staff. Table 2 shows the variables used as inputs and out- 
puts, as well as the environmental variables. The input and 
output variables as well as the environmental variables 
were chosen taking into account the literature review and 
the available data. All the environmental variables were 
collected by analyzing the activity reports of the ACES. 

Data envelopment analysis 

In order to study efficiency, we applied DEA, a non- 
parametric full frontier method based on linear program- 
ming technique with data from two consecutive years. 
This technique is used with the aim of evaluating the effi- 
ciency of the Decision Making Units (DMUs), by analysing 
the optimal combinations between inputs and outputs, i.e. 
between consumed resources and the resulting services or 
goods [25]. This methodology optimizes each individual 
observation and builds a production frontier, constituted 
by the efficient DMUs. Several models might be applied 
and choices regarding the type of return to scale and the 
orientation must be made in agreement with the produc- 
tion process that is being analyzed [26]. Concerning the 
orientation, we decided to use the following three orienta- 
tions to compare the results and because in healthcare we 
can consider both points of view: 

• Input orientation: this orientation is used when it is 
assumed that we have more control of the inputs, it is 
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Table 2 Inputs and Outputs used in the DEA models to measure efficiency and environmental variables 



Inputs 


Outputs 


Environmental variables 


x1 : doctors' working hours 


y1 : number of adult health consultations 


EV] : Population 


x2 : nurses' working hours 


y2 : number of speciality consultations 


EV2 : Population density 


x3 : administrative staff working hours 


y3 : number of urgency consultations (SAP, 
CATUS) 


EV3 : Percentage of patients aged 65 years 
old or older 




y4 : number of home visits by doctors 


EVA : Mortality Rate 


x4 : total costs 


y5 : total number of consultations 


EV5 : Percentage of patients without 
designated doctor 

EV6 : Distance to the nearest hospital 




y6 : number of group education sessions 


EV7 \ Purchasing power 




y7 : number of consultations by nurses 






y8 : number of injections, vaccinations, curatives 
and other treatments 






y9 : number of home visits by nurses 






yl 0 : total number of nursing services 






yl 1 : total number of public health activities 





also intended to emphasize the reduction of excessive 
inputs and we think it is possible to reach the same 
outputs with fewer inputs [27]. 

• Output orientation: the output-oriented model is 
adopted if one considers to have more control on the 
outputs, for instance, by controlling the reputation or 
the quality of service and when it is considered to be 
possible to increase the outputs, without any 
proportional change in the inputs [27]. 

• Non-oriented model: this model considers to be 
possible to reduce the inputs and simultaneously 
increase the outputs. 

In this study, we used three possible orientations: input- 
orientation, output-orientation and also a non-oriented 
model, the additive one, which is based on slacks, exces- 
sive inputs or missing outputs that exists even after the 
proportional change in the input or the outputs [27]. 
These orientations were used in connection with both 
constant returns to scale (CRS) and variable returns to 
scale (VRS) technologies. 

In the interest of understanding why both CRS and 
VRS technologies were used, it is important to clar- 
ify both concepts. CRS technology assumes that scale 
of economies do not change as the size of the service 
increases while VRS assumes that scale of economies do 
not change with the size of the service. Thus, CRS tech- 
nology is not present if a proportional increase in one 
input can cause greater than a proportionate increase in 
output. In healthcare, not all of the DMUs are producing 
at the same scale, which might be due to many factors, 



such as financial constraints, technical constrains, or poor 
organization. 

The method was chosen depending on the perspec- 
tive of the analyst. If we believe that the DMU is 
not operating at the optimal scale, the VRS technol- 
ogy is recommended. However, by calculating both 
efficiencies, we can determine a third efficiency mea- 
sure: the scale efficiency, which is defined as the ratio 
between the efficiency computed with between VRS and 
CRS: 

Q scale = QVRS/OCRS (1) 

The efficiency scores (0) for the DMUs (j = l,...,n) are 
computed for the selected outputs (y r j) and inputs fey). 
We used the dual model in order to observe benchmarks 
and their weights (A.). The (A.) values allow us to under- 
stand the return to scale for each DMU. If EA. = 1, it 
means that the DMU is working with CRS. If EA. < 1, 
the DMU is working with increasing returns to scale. 
Finally, if EA. > 1, the decreasing return to scale is 
the technology. In the objective function (e) is called the 
non-Archimedean, which is defined as infinitely small. 
s~ and 5 + represent input and output slacks, respectively 
(Table 3). 

In contrast with the parametric techniques, such as the 
Stochastic Frontier Analysis, the non-parametric tech- 
nique of DEA seems to be a better option since it does not 
require the specification of the functional form that links 
the inputs to the outputs. 
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Table 3 Mathematical formulation of the different models and orientations [27] 



Orientation 



CRS 



VRS 



Inputs 



Minimize 6 - £(E S / + E s f) subject to 1a. 

/=1 r=l 

n 

Etyty + s /~ — ^* x '0 witn ' — 2a. 

7=1 
n 

J2 ^jYrj — st = Yro with r = 1 , . . . , s 3. kj > 0 with 

7=1 

7 = 1 n 



Minimize 6 - s(J2 s i + E subject to 1 a, 2a, 

/=! r=1 

n 

3 and 4. J] A 7 - = 1 

7=1 



Outputs 



Maximize 4> - £(E S / + E subject to 1b. 

i=] r=1 

n n 

J2 ^jXij+s~ = Xio with /' = 1 , . . . , m 2b. J2 tyrj ~ 

7=1 7=1 

s+ = (py r0 with r — 1 , . . . , s 3. kj > 0 with j = 
1,...,n 



Maximize 4> -s(J2 s,- + J] s+) subject to 1 b, 2b, 

/=! r=1 

3 and 4. £ Ay = 1 

7=1 



Non-oriented 



m s 



Maximize J2 s i + J2 s t subject to 1 c. J2 h x >i + 



7=1 



Sj = x i0 i = 1,...,m 2c. Y,kjyrj - s+ 



7=1 



y r0 r = 1,. . .,s 3. kj > 0 j =],..., n 



E 5 ," + E s r + subject to 1 c, 2c, 3 and 4. kj = 1 



Adopted models 

In the light of data constraints, 4 models were used: 
medicine, nursing, global and global with total costs. They 
are presented next: 

Model I (Medicine): it includes all the ACES' medical 
services. The input used was xl and the outputs were yl, 
y% y3 and j/4; 

Model II (Nursing): it comprises all the ACES' nursing 
services. The input adopted was x2 and the outputs were 
3/6, y7, 3/8 and y9; 

Model III (Global): this model aims to include all the 
ACES' services. The input used was xl, x2 and x3 and the 
outputs were y5, ylO and yll; 

Model IV (Global with total costs): model similar to 
Model III. However, the input adopted was j/4 (total costs), 
instead of staff number resources. The outputs used were 
y5, ylO and 

These models were used in order to include the two 
main services of the ACES (medicine and nursing service) 
and to calculate the global efficiency. The global efficiency 
is calculated by using two different models because the 
data regarding the costs was only available for the second 
year of study. 

The order-m methodology 

The environmental variables are exogenous factors that 
cannot be categorized as inputs or outputs but might 
affect efficiency [28] . As a result, they are not controlled 
by the DMU managers. It is extremely important to take 
these variables into account. If not, biased conclusions 
might be drawn. Although there is no agreement on the 
best technique to be used, in this study we used a recent 
methodology: the order-m [28], because, being a partial 
frontier method, order-m has proven to be less sensitive 



than DEA to extreme values and able to overcome the 
deterministic nature of traditional non-parametric tech- 
niques [28]. According to the order-m methodology, the 
production process can be described by the joint probabil- 
ity of measure of <^(X, Y), on space £R^_ x according 
to 2. 



H X Y(x,y) = Prob(X < x, Y > y) 



(2) 



where X are the inputs and Y are the outputs. 

Afterwards, in an input orientation context, the like- 
lihood function can be decomposed into two according 
to Bayes rule (3), and then, efficiency can be computed 
(4, 5, 6). 



H XY (x,y) = Prob(X < x\Y > y)Prob(Y > y) 
= F X \Y(x\y)S Y (y) 

0(x,y) = mf{0\F X \Y(Ox\y)>0} 
= mf {0\HxY(Px,y) > 0} 



§(x,y) = \nf {0\F X \Y(0x\y) > 0} 



n,n( x >y) = j (l-Fx\Y,n(ux\y)) m tiu 
0 



(3) 



(4) 



(5) 



(6) 



J2l(Xi<ux,Yi>y) 

with Fx\Y,n(ux\y) = „ and I(k) being the 

EnYi>y) 
i=l 

indicator function that take the value I(k) = 1 when k is 
true or I(k) = 0 otherwise. 

According to Daraio and Simar [28], the inclusion of 
exogenous variables can easily be done by limiting the 
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production process to a given value of the exogenous 
variable (7, 8). 

Hxrfay) = Fx\Y,z(x\y,z)S Y \z(y\z) (7) 

oo 

O m (x,y\z) = j(l -F X \Y,z,n(ux\y>z)) m du (8) 

0 

^I{Xi<uxJi>y)K(^^) 

where F X \Y,z,n( ux \y> z ) = » 77777 > * is tne 

i=l v 7 

bandwidth and 7<T(») is the kernel function. 

With the aim of analyzing the influence of the 
exogenous variable on the production process, a non- 
parametric smoothed regression of the ratios between the 
order-m conditional efficiencies and the unconditional 
efficiencies is applied. If the regression has a positive 
slope, the exogenous variable has a negative effect on the 
production process because the environmental variable 
acts like an "undesirable" output to be produced, requir- 
ing the usage of more inputs [28] . Otherwise, it will have 
the opposite effect. 

The Kruskal-Wallistest 

An important study regarding the PHC is the analysis of 
the horizontal equity, which concerns fairness or justice in 
the treatment and measures whether patients from differ- 
ent groups have similar access to the services they equally 
need [13]. 



In order to analyze the horizontal equity of access 
regarding the NUTS III we studied the relationship 
between the NUTS III and (1) the efficiency and (2) the 
percentage of patients without a designated doctor. To 
test the hypothesis of relevant differences in equity, we 
used the non-parametric Kruskal-Wallis test, with multi- 
ple comparisons. 

The Kruskal-Wallis test is a non-parametric method 
used to determine whether three or more independent 
groups are the same or different on some variable of inter- 
est [29]. It calculates the probability of being wrong when 
concluding that there is no difference between three or 
more treatment groups [30] . 

The test will assume that the samples are identical 
and check if the differences found between the groups 
are genuine and if there are evidences to reject the null 
hypothesis. 

The ANOVA test is more powerful if we are studying a 
normal distribution. However, since we are using a non- 
parametric method, the Kruskal-Wallis is more appropri- 
ate because it does not assume that kind of distribution. 

Results 

Sample characteristics 

The summary statistics for the variables used in the 
research are described in Table 4. 

Efficiency and environmental variables 

Both CRS and VRS technologies concerning input ori- 
entation, output orientation and the non-oriented model 
were used in order to compare the results from each 



Table 4 Summary statistics of inputs and outputs (year 201 0) 



Variables 


Average 


Evolution 
(2009 to 2010) 


Median 


Std. deviation 


Maximum 


Minimum 


X] 


3583 


-3.9% 


3596 


1191 


6009 


1750 


x2 


3537 


-1.6% 


3693 


706 


4655 


2170 


x3 


6116 


-5.8% 


5828 


1709 


9905 


2660 


xA 


34147052 


ND 


34817334 


7503483 


44872810 


17898835 


yi 


351901 


1 .8% 


355779 


79532 


517515 


189954 


y2 


75365 


-0.1% 


73766 


26072 


158988 


39745 


y3 


47693 


-24.24% 


47052 


32488 


105067 


2503 


y4 


1879 


3.4% 


1788 


932 


4976 


576 


ys 


476837 


-1.9% 


451655 


109922 


685802 


276495 


y e 


918 


-5.3% 


726 


868 


1804 


1 


y7 


212289 


9.3% 


181902 


80977 


485108 


1 07244 


y8 


173392 


0.7% 


147439 


73976 


317073 


78673 


y9 


50368 


4.0% 


42213 


21870 


103511 


28695 


y10 


436967 


5.1% 


396031 


148284 


842544 


237951 


y11 


13907 


-0.9% 


10060 


10309 


39012 


3876 
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approach. Tables 5, 6 and 7 summarize the efficiency 
results for the three kinds of orientation. 

Tables 8, 9 and 10 shows the X results analysis, i.e., the 
benchmarks and also the returns to scale, as explained in 
Subsection 'Data envelopment analysis'. 

Regarding the environmental variable, the graphic 
results of the order-m regression are illustrated in 
Figures 2 and 3. 

Equity of access 

Figures 4 and 5 show the results of the first Kruskall- 
Wallis test: the relationship between the NUTS III and the 
efficiency. 

Figure 6 shows the results of the second test: the rela- 
tionship between the NUTS III and the percentage of 
patients without a designated doctor. 

Quality 

In addition to the efficiency and the equity of access, it is 
also of utmost importance to evaluate the quality of the 
services provided. According to Fornell [31], the patients' 
level of satisfaction on the PHC is very important to track 
the quality of service. Therefore, this analysis uses the 
ratio between the complaints and the total of ACES' activ- 
ities. These data are reported in the Project SimCidadao 



report and can be considered a good indicator of the 
patient s level of satisfaction. Figure 7 shows the trade-offs 
between the VRS efficiency and the ratio addressed above, 
for the years 2009 and 2010. 

Discussion 

Efficiency and the influence of the environment on the 
efficiency 

By analyzing the general efficiency results referred to in 
Subsection 'Quality' (Tables 5, 6 and 7 some comments 
can be made. In general, we can see that the results were 
very similar for all types of used orientation. 

Regarding the results themselves, the ACES 17 (Oeste 
Norte) is one of the most efficient. As can be seen in 
Table 5, this ACES is referred to 7 times as the most 
common benchmark, i.e. the best practice for the other 
ACES. In fact, Model I (CRS) is the only model where this 
ACES does not provide a unitary efficiency. ACES 2 and 
11 are the least efficient. In fact, these ACES have poor 
results regarding contractualization. The model with the 
lowest efficiency is Model II (Nursing), where the num- 
ber of scores below 60% that can be found is maximum. 
A general improvement of efficiency between 2009 and 
2010 can be observed. For instance, in 2009, VRS showed 
a result of 0.944 for Model III, while in 2010, the same 



Table 5 Efficiency results for 2009 and 201 0 (input orientation) 

Model I Model II Model III Model IV 



Year 


2009 


2010 


2009 


2010 


2009 


2010 


2010 


6 crs average 


0.793 


0.816 


0.764 


0.769 


0.913 


0.935 


0.918 


min6b?s 


0.581 


0.569 


0.469 


0.465 


0.761 


0.817 


0.745 


DMU with min Oqrs 


2 


2 


21 


11 


11 


11 


2 


Number of efficient DMUs (CRS) 


4 


6 


5 


6 


8 


8 


6 


DMUs with 9 C rs > 95% 


4 


6 


6 


7 


9 


11 


12 


DMUs with 6crs < 60% 


2 


1 


5 


3 


0 


0 


0 


Most common benchmarks (CRS) 


22 


22 


17 


17 


17 


17 


8 


Ovrs average 


0.852 


0.857 


0.839 


0.849 


0.944 


0.961 


0.928 


min 0 V rs 


0.585 


0.595 


0.581 


0.520 


0.775 


0.835 


0.754 


DMU with min Ovrs 


11 


2 


21 


15 


11 


15 


2 


Number of efficient DMUs (VRS) 


7 


8 


8 


7 


11 


13 


9 


DMUs with 0 V rs > 95% 


9 


8 


8 


9 


13 


15 


12 


DMUs with Ovrs < 60% 


1 


1 


1 


1 


0 


0 


0 


Most common benchmarks (VRS) 


15 


13e22 


5 


17 


17 


13 e 17 


14 


Oscaie average 


0.935 


0.954 


0.905 


0.903 


0.968 


0.972 


0.990 


min Oscale 


0.722 


0.783 


0.690 


0.622 


0.837 


0.848 


0.960 


DMU with min 0s ca ie 


15 


17 


9 


9 


12 


5 


9 


Number of scale efficient DMUs 


4 


6 


5 


6 


8 


8 


7 


DMUs with Oscale > 95% 


11 


17 


8 


10 


17 


17 


22 


DMUs with Oscale < 60% 


0 


0 


0 


0 


0 


0 


0 
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Table 6 Efficiency results for 2009 and 201 0 (output orientation) 





IVIUUCI 1 




Mnripl II 

IVIUUCI II 




Model III 


MoHpI IV 

IVIUUCI 1 V 


Year 


2009 


2010 


2009 


2010 


2009 


2010 


2010 


Or d<; averaae 


1,3010 


1,264 


1,383 


1,380 


1,104 


1,076 


1,099 


max 6crs 


1,722 


1,758 


2,132 


2,149 


1,314 


1,224 


1,343 


DMU with max Ocrs 


2 


2 


21 


1 1 


11 


11 


2 


Number of efficient DMUs (CRS) 


4 


6 


5 


6 


8 


8 


6 


DMUs with Oqrs < 1,05 


4 


6 


6 


7 


9 


11 


12 


DMUs with BroQ > 1 6 

L/l V IU J Will 1 1 / ^ 


3 


3 


6 


5 


0 


0 


o 


Most common Benchmark 


22 


22 


17 


17 


17 


17 


8 


A/dc averaae 


1,154 


1,134 


1,307 


1,292 


1,049 


1,037 


1,084 


|T|3Y A/nr 
I I iua <^Vr\b 


1,585 


1,590 


2,1 25 


2,01 7 


1,306 


1,188 


1,3367 


DMU with max 6V/?s 


5 


5 


21 


1 1 


11 


11 


2 


Number of efficient DMUs (VRS) 


7 


8 


8 


7 


11 


13 


9 


DMUs with Ovrs < 1,05 


10 


10 


8 


9 


14 


15 


12 


DMUs with Ovrs > 1,6 


0 


0 


4 


4 


0 


0 


0 


Most common Benchmark 


17 


13 


17 


17 


17 


17 


8e 14 


Oscaie average 


1,132 


1,115 


1,062 


1,070 


1,053 


1,037 


1,014 


max 6> Sco/e 


1,432 


1,412 


1,449 


1,607 


1,189 


1,179 


1,094 


DMU with max 9s ca ie 


15 


1 


9 


9 


1 


5 


15 


Number of scale efficient DMUs 


4 


6 


5 


6 


8 


8 


6 


DMUs with Oscaie < 1,05 


9 


10 


13 


12 


14 


16 


21 


DMUs with Oscaie > 1,6 


0 


0 


0 


1 


0 


0 


0 


technology showed a score of 0.96. One of the possible 
reasons is the start-up of 10 new USFs. Also, 2009 was the 
foundation year of the ACES and it is likely that they were 
not as organized as in 2010. 

Regarding the returns to scale, some important conclu- 
sions can also be drawn. By observing Table 8 (and also 
Tables 9 and 10), it can be seen that as far as Model I 
is concerned, more than half of the ACES are working 
with decreasing returns to scale, i.e. they are operating 
above the optimal scale. ACES 1 and 3 always present 
decreasing returns to scale for Model I. Actually, both 
ACES are serving a population over the theoretical limit 
of 200 000. Probably the effect of diseconomies of scale 


would decrease if the 3 ACES of the Lisbon area (Lisboa 
Norte, Lisboa Oriental and Lisboa Central) were split into 
4. In contrast, in Model II (nursing), more than 50% of the 
ACES have increasing returns to scale. This means that 
concerning the variables studied, in these ACES nursing is 
operating below the optimal scale. 

As stated before, adjusting for the environment in 
efficiency studies is very important. In this paper, the 
exogenous variables considered are the ones described 
in Table 3, respectively population density, percentage of 
patients aged 65 years old or older, mortality rate, percent- 
age of patients without a designated doctor, distance to the 
nearest hospital and purchasing power. 


Table 7 Efficiency results for 2009 and 201 0 (non-oriented model) 












Model 1 


Model II 


Model III 


Model IV 


Year 


2009 


2010 


2009 


2010 


2009 


2010 


2010 


Number of efficient DMUs (CRS) 


4 


6 


5 


6 


8 


8 


6 


Number of non efficient DMUs (CRS) 


18 


16 


17 


16 


14 


14 


16 


Most common Benchmark 


22 


22 


17 


17 


17 


17 


8 


Number of efficient DMUs (VRS) 


7 


8 


8 


7 


11 


13 


9 


Number of non efficient DMUs (VRS) 


15 


14 


14 


15 


11 


9 


13 


Most common Benchmark 


13 


13 


17 


17 


17 


17 


14 
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Table 8 General statistics concerning returns to scale (input orientation) 





Model 1 




Model II 






Model III 


Model IV 


Year 


2009 


2010 


2009 


2010 


2009 


2010 


2010 


Y!iXcrs average 


1 .obo 


I .j I U 


U./zo 


n q~)q 


U.OD/ 


n Q/i ^ 

u.y4j 


U.y4z 


max £A.o?s 


3.506 


2.941 


1.137 


1.284 


1.123 


1.488 


1.316 


DMU with max EA.c/?s 


3 


3 


1 


16 


21 


11 


16 


#of DMUs with EA. C /?s > 1 


14 


12 


2 


4 


3 


6 


6 


min £A.q?s 


0.862 


0.941 


0.362 


0.486 


0.473 


0.518 


0.523 


DMU with min EA. C /?s 


12 


5 


9 


9 


9 


5 


9 


#of DMUs with £A.a?s < 1 


4 


4 


15 


12 


11 


8 


10 


#of DMUs with £A.a?s = 1 


4 


6 


5 


6 


8 


8 


6 



Since the sample includes only ACES from LVT, without 
considering other important variables, such as laws and 
policy administration, they can be considered relatively 
homogeneous. It is important to highlight the meaning 
of variable 6, the distance to the reference hospital This 
distance refers to the average of the distances between 
the head office of each health center and the reference 
hospital. 

The discussion of the results obtained by using the 
order-m methodology is summarized below. 

Concerning the population, efficiency decreases with 
the increase of the population. This result suggests that 
the population acts as an output, requiring more inputs in 
the production process. This is in agreement with the sug- 
gestion made before, about splitting the 3 ACES of Lisbon 
into 4. 

As to the population density, the results show that there 
is no influence on efficiency. 

About the percentage of patients aged 65 years old or 
older, there is a negative influence of this variable on effi- 
ciency. In fact, aging leads to a significant rise in the need 
for community and social care. Consequently, efficiency 
decreases due to the increase of costs associated with 
aging. 



Mortality shows no influence on efficiency. Avoidable 
mortality could have been used instead. However, there 
were no data available on this variable. 

As far as the percentage of patients without a desig- 
nated doctor is concerned there is no influence of this 
variable on efficiency, maybe because of the reorganiza- 
tion of some of the functional units of the ACES, namely 
USFs and UCSPs. 

The variable distance to the reference hospital has a 
positive influence on efficiency. This is not surprising, as 
when a hospital is located farther away from a population, 
more patients will go to a PHC. 

Regarding purchasing power, this variable has a negative 
influence on the efficiency. Actually, patients with higher 
purchasing power tend to have health insurance and use 
the PHC less. Furthermore, patients with less purchasing 
power cannot afford private health care and need to resort 
to PHC. 

Equity of access 

By observing Figures 4, 5 and 6, we see that both tests 
suggest that there is no evidence of relevant differences, 
and consequently there is horizontal equity regarding the 
NUTS III. In fact, despite some demographic inequalities, 



Table 9 General statistics concerning returns to scale (output orientation) 







Model 1 


Model II 


I 


Model II 


1 


Model IV 


Year 


2009 


2010 


2009 


2010 


2009 


2010 


2010 


HkcRs average 


1,798 


1,672 


2,038 


1,520 


0,938 


1,013 


1,028 


max £A.o?s 


3,721 


3,433 


7,018 


3,207 


1,288 


1,822 


1,465 


DMU with max £A.a?s 


3 


3 


14 


14 


15 


11 


1 


#of DMUs with £A. C /?s > 1 


18 


16 


15 


15 


6 


7 


10 


min £A.o?s 


1 


1 


0,884 


0,781 


0,483 


0,581 


0,544 


DMU with min £A.a?s 






9 


9 


9 


12 


9 


#of DMUs with Y,k C Rs < 1 


0 


0 


2 


1 


8 


7 


6 


#of DMUs with £A. C /?s = 1 


4 


6 


5 


6 


8 


8 


6 
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Table 1 0 General statistics concerning returns to scale (non-oriented model) 



Year 


Model 1 




Model II 


Model III 


Model IV 
2010 


2009 


2010 


2009 


2010 


2009 


2010 


YiXcrs average 


1,832 


1,761 


n QQ 1 ") 

u,y& i z 


u,yoo 


0,939 


0,932 


n Q/i 1 q 

u,y4 1 o 


max £A.o?5 


3,581 


3,293 


1,378 


1,406 


1,306 


1,207 


1,316 


DMU with max £A. C /?s 


3 


3 


18 


3 


15 


15 


16 


#of DMUswith ZXcrs > 1 


18 


16 


8 


5 


6 


5 


6 


min £A.o?s 






0,537 


0,5345 


0,483 


0,5628 


0,5225 


DMU with min SA. C /?s 






9 


9 


9 




9 


#of DMUswith £A. C /?s < 1 


0 


0 


9 


11 


8 


9 


10 


#of DMUswith EA. CT s = 1 


4 


6 


5 


6 


8 


8 


6 
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150000 200000 250000 12 14 16 18 20 22 24 

Population Percentage of patients aged greater than or equal to 65 years 




all the municipalities have healthcare facilities. The USFs 
can contribute to improve equity because one of their 
main goals is to have close contact with the population 
and to avoid having patients without a designated doctor 
[32]. 

These results go in line with the study from a public 
authority (Alto Comissariado da Saude [33]). This study 
states that an access improvement, a higher number of 
consultations and a bigger rational of healthcare have been 
observed since 2009. 

Quality 

Figure 7 shows a large number of complaints in Grande 
Lisboa and Peninsula de Setubal. However, this behavior 
might be a consequence of patients' higher expectations. 



Because they have many additional options, they might 
expect a different kind of service. Also, in Figure 7 some 
particular results should be noticed. There is a higher inci- 
dence of complaints for ACES Algueirao - Rio de Mouro 
(9) in 2009 and for ACES Leziria II (22) in 2010. Regarding 
the ACES Leziria II (22), an increase of about 200% was 
recorded, compared with 2009. This increase may be due 
to the lack of doctors and the excess work load of the doc- 
tors working in this ACES. The reason why in the ACES 
Algueirao - Rio de Mouro (9) there was a higher incidence 
of complaints is probably the same. In fact, this is the 
ACES with higher average number of patients per doctor. 
The ACES Cascais (11) and Arco Ribeirinho (15) can be 
found in the region of high complaints and low efficiency, 
for both years. The high level of complaints of ACES 15 
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may be related to the average waiting time for a consulta- 
tion in a UCSP, which is typically greater than the average 
waiting time in a USE 

Finally, the ACES Oeste Norte (17) presents one of the 
highest patient levels of satisfaction. This ACES is the one 
that can be used as a benchmark for most of other ACES, 
as previously mentioned. This idea is also corroborated by 
the fact that the average number of patients per doctor is 
low. 

It is important to stress that the rate of complaints is 
only an indicative measure of the patients' satisfaction 
level. The rate of complaints might be used as a guide- 
line to satisfaction and quality, but satisfaction surveys 
would have been better in our view. Although ARS has 
been trying to carry out these surveys, they are currently 
suspended as a result of the current economic conditions. 

Conclusions 

Taking into account the pressure to decrease costs with 
healthcare, the efficiency analysis of PHC is of paramount 



importance. However, besides the problems with getting 
data, the evaluation of PHC is still a difficult and con- 
troversial task. Nonetheless, during this analysis, some 
important conclusions were drawn. 

In terms of efficiency, on the whole, there was a general 
improvement in efficiency between 2009 and 2010. It was 
found that nursing was the service with lowest scores. In 
particular, the ACES Oeste Norte (17) was scored as one 
of the most efficient and identified as a unit to be used as 
benchmark. On the other hand, ACES Lisboa Oriental (2) 
and Cascais (11) were scored as the least efficient. With 
regard to returns to scale, it would be a good policy to split 
the 3 ACES from Lisbon (Lisboa Norte, Lisboa Oriental 
and Lisboa Central) into 4. Regarding equity of access, the 
objectives of ensuring equal opportunity to all the patients 
and the allocation of resources and services in a fair, con- 
sistent and inclusive manner appears to be working well, 
at least according to the two tests performed. 

Concerning quality versus efficiency, the ACES which 
can more successfully combine both these aspects are 
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the ACES Zezere (20), Ribatejo (21) and also the ACES 
Oeste Norte (17), previously referred to as a benchmark. 
Conversely, the most problematic cases are the ACES 
Cascais (11) and Arco Ribeirinho (15), both located in the 
region of high complaints and low efficiency. Other exam- 
ples of problematic cases are the ACES Algueirao - Rio de 
Mouro (9) and Leziria II (22). These ACES are efficient 
only because they have a low level of human resources 
and because of that they receive massive complaints. As 
previously mentioned, satisfaction surveys regarding PHC 
are, at least for the time being, suspended. We intend 
to include them in this study as soon as they become 
available. 

Finally, regarding the exogenous variables, the distance 
to the nearest hospital has a positive influence on effi- 
ciency. In contrast the order-m test showed that the pur- 
chasing power, the percentage of patients aged 65 years 
old or older and the population affect efficiency negatively. 

In order to strengthen this study, it should be repeated 
with more information and more robust data in the future. 
For instance, it would be important to include more data 
regarding costs and health results, such as prevention 
and control of diabetes. The inclusion in the study of 
economies of scope would also be a good complement 
to this work. Also, it would be interesting to extend 
this study to all the ACES and even compare the effi- 
ciency of PHC before and after the reform of the PHC 
structure. 

Nevertheless, despite some difficulties and some work 
that can still be done, this research is important to under- 
stand how the ACES have been performing in their 
first years of activity. This study shows that by estab- 
lishing a learning network there is room for improving 
the efficiency of ACES with a better usage of the avail- 
able resources whilst simultaneously keeping quality and 
equity. 

Finally, we should note that an important aspect in this 
kind of studies is the fact that they allow the assessment 
of the current structure of the PHC in the country while 
taking into account not only the size but also other factors 
which are important for their performance, such as the 
proximity of hospitals. 
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